Concrete Dropout

Dropout is used as a practical tool to obtain uncertainty estimates in large vision models and reinforcement learning (RL) tasks. But to obtain well-calibrated uncertainty estimates, a grid-search over the dropout probabilities is necessary - a prohibitive operation with large models, and an impossible one with RL. We propose a new dropout variant which gives improved performance and better calibrated uncertainties. Relying on recent developments in Bayesian deep learning, we use a continuous relaxation of dropout's discrete masks. Together with a principled optimisation objective, this allows for automatic tuning of the dropout probability in large models, and as a result faster experimentation cycles. In RL this allows the agent to adapt its uncertainty dynamically as more data is observed. We analyse the proposed variant extensively on a range of tasks, and give insights into common practice in the field where larger dropout probabilities are often used in deeper model layers

Characterization of Chicken Tumor Necrosis Factor-alpha, a Long Missed Cytokine in Birds

Tumor necrosis factor-alpha (TNF-alpha) is a pleiotropic cytokine playing critical roles in host defense and acute and chronic inflammation. It has been described in fish, amphibians, and mammals but was considered to be absent in the avian genomes. Here, we report on the identification and functional characterization of the avian ortholog. The chicken TNF-alpha (chTNF-alpha) is encoded by a highly GC-rich gene, whose product shares with its mammalian counterpart 45% homology in the extracellular part displaying the characteristic TNF homology domain. Orthologs of chTNF-alpha were identified in the genomes of 12 additional avian species including Palaeognathae and Neognathae, and the synteny of the closely adjacent loci with mammalian TNF-alpha orthologs was demonstrated in the crow (Corvus cornix) genome. In addition to chTNF-alpha, we obtained full sequences for homologs of TNF-alpha receptors 1 and 2 (TNFR1, TNFR2). chTNF-a mRNA is strongly induced by lipopolysaccharide (LPS) stimulation of monocyte derived, splenic and bone marrow macrophages, and significantly upregulated in splenic tissue in response to i.v. LPS treatment. Activation of T-lymphocytes by TCR crosslinking induces chTNF-alpha expression in CD4(+) but not in CD8(+) cells. To gain insights into its biological activity, we generated recombinant chTNF-alpha in eukaryotic and prokaryotic expression systems. Both, the full-length cytokine and the extracellular domain rapidly induced an NF kappa B-luciferase reporter in stably transfected CEC-32 reporter cells. Collectively, these data provide strong evidence for the existence of a fully functional TNF-alpha/TNF-alpha receptor system in birds thus filling a gap in our understanding of the evolution of cytokine systems